Why experiments?

• Experiments help establish causation.

• Experimental research has real policy implications.

What are some problems with experiments?

• Experiments can be difficult to run and expensive.

• Many are unethical.

• Labratory.

What is a lab experiment?

• Expose subjects to treatment in controlled (artificial) setting and observe reactions.
• Complete control of environment (very high internal validity).
• Does the lab replicate a 'real world' environment? (lower external validity).

What are some examples?

• Stanford Prison Experiment, Philip Zimbardo.
• Effects of 'irrelevant stimuli' on responses to policy prompts about immigration.

What did this experiment involve?

• 224 subjects read and responded to policy statements. Eg, asked to respond to the statement: 'The number of illegal immigrants coming to the United States will drastically increase in six years.' Responses rated as positive or negative.
• Subjects’ responses to policy prompts tended to reflect preexisting views.
• However, difference dwarfed by difference between subjects with similar views exposed to different 'irrelevant stimuli'.
• Exposure to positive stimuli = 42% fewer negative and 160% more positive thoughts about illegal immigration than negative.

What were these 'irrelevant stimuli'?

• This is what makes this an experiment.
• Random treatment: Before seeing each policy statement, each subject was subliminally exposed (for 39 milliseconds, well below the threshold of conscious awareness) to: a smiling, frowning or neutral cartoon face.
• Quick exposure to images of cartoon smiley faces had a larger effect than prior immigration attitudes on the valence of thoughts in response to illegal immigration policy prompts.

Source: Washington Post

What is a lab experiment?

• Expose subjects to treatment in controlled setting and observe reactions.
• Complete control of environment (very high internal validity).
• Does the lab replicate a 'real world' environment? (lower external validity).

What are some examples?

• Stanford Prison Experiment, Philip Zimbardo.
• Effects of 'irrelevant stimuli' on responses to policy prompts about immigration.
  • Problems with these kinds of research design?

• Labratory.
• Field.

What is a field experiment?

• Research in a realistic setting. Treatment intervenes and manipulates independent variable(s) and controls situatation as well as possible.
• High external validity.
• Less control of environment, leading to lower internal validity.

What are some examples?

• The impact of school lunches on performance.
  
• AB testing for websites.
• Campaign advertising.

• In 2005 Republican consultant Dave Carney hired political scientist Donald P. Green to run a series of random trials for Texas Governor Rick Perrys (primary) re-election campaign.
• Spent $2 million in positive, randomly assigning television and radio ads, to 18 different markets over three weeks.
• Found short-term effect of TV ads: increased Perrys numbers by 5% for ~ a week.

• Labratory
• Field
• Survey

What is a survey experiment?

• Can be a subset of a field and lab experiment.
• Apply a treatment on a random subset of the sample then ask a series of questions. The random treatment is what makes these experiments.
• Treatment can be another question.

For instance. Want to understand the causal impact of economic insecurity on support for populism.

Are you extremely worried, very worried, moderately worried, a little worried or not at all worried about:

1. Losing your job in the next 12 months.
2. Having enough money to retire on.
3. Paying your rent or mortgage.

We then ask a series of questions designed to tap into populist or authoritarian attitudes, such as:

Do you strongly agree, agree, disagree or strongly disagree that:

1. Politicians need to follow the will of the people.
2. Most politicians are competent people.
3. The elite have different political views to ordinary people.

<1> Strongly agree <2> Agree <3> Disagree <4> Strongly disagree

What is a survey experiment?

• Can be a subset of a field and lab experiment.
• Apply a treatment on a random subset of the sample then ask a series of questions. The random treatment is what makes these experiments.
• Treatment can be another question.

• Also can be statements or media, or even ordering of questions.
• List experiments arguably a subset of this.

Problems with survey experiments?

• Labratory.
• Field.
• Survey.
• Natural.

What is a natural experiment?

• Using a treatment that is 'random' but not controlled by the researcher to understand causality.
• Lotteries, natural disasters, Media markets.

What is a natural experiment?

• Using a treatment that is 'random' but not controlled by the researcher to understand causality.
• Lotteries, natural disasters, Media markets.

Problems with natural experiments?

• We don't control the application of the treatment.
• The treatment may not be truly exogenous to the outcome we are studying.

• We want to understand human behavior, and processes involving humans and their actions, predict these behaviours and processes, and find solution to problems that face society.
• Much of human behaviour can be understood geographically.
• Tobler’s First Law of Geography; spatial dependence. Basis of much spatial analysis.
• Spatially defined groups (towns, cities, states, countries) often have a great deal of social, historical and economic meaning. Not merely defined by their distance from one another.

• Spatial analysis examines data in cross-section, as opposed to longitudinal analysis (although the two can be combined).
• There is often a great deal of data on spatially defined groups (towns, cities, states, countries). Ie from census.
• Spatial analysis can play important role in inductive and deductive approaches to science. Inductive: display of data in spatial context may reveal patterns and anomalies and suggest processes explaining them (ie John Snow, 1854).

• Spatial analysis examines data in cross-section, as opposed to longitudinal analysis (although the two can be combined).
• There is often a great deal of data on spatially defined groups (towns, cities, states, countries). Ie from census.
• Spatial analysis can play important role in inductive and deductive approaches to science. Inductive: display of data in spatial context may reveal patterns and anomalies and suggest processes explaining them (ie John Snow, 1854).
• Deductive approach to test theories and principles. Recognition of confounding factors.

• Tables.

• Tables.
• Graphs.

• Tables.
• Graphs.
• Maps.

• Standard.
• Choropleth.
• Distorted.

Source: Mark Newman

Source: Mark Newman

However, this kind of map does not always work

However, this kind of map does not always work

Source: Washington Post

However, this kind of map does not always work

Source: Washington Post

Source: Mark Newman

• Standard.
• Choropleth.
• Distorted.
• Dot.

Source: Kenneth Field

• Standard.
• Choropleth.
• Distorted.
• Dot.
• Technical advances (GIS in particular, and interactive).

Dr Shaun Ratcliff
Email: shaun.ratcliff@sydney.edu.au
Office: Room N416, Institute Building (H03)

Nick James (Friday lab)
Email: nicholas.james@sydney.edu.au
Office: Level 5, The School of Information Technologies, Building J12